This adds support for encoding and parsing components which use the [Async
ABI](https://github.com/WebAssembly/component-model/blob/main/design/mvp/Async.md)
and associated canonical options and functions, along with the [`stream`,
`future`, and `error`](WebAssembly/component-model#405)
types.

Note that the `error` type was recently (about 30 minutes ago) renamed to
`error-context` in Luke's spec PR.  I haven't updated this implementation to
reflect that yet, but will do so in a follow-up commit.  That should allow us to
avoid conflicts with existing WIT files that use `error` as a type and/or
interface name.

This does not include any new tests; I'll also add those in a follow-up commit.

See bytecodealliance/rfcs#38 for more context.

Signed-off-by: Joel Dice <[email protected]>

This adds support for generating bindings which use the [Async
ABI](https://github.com/WebAssembly/component-model/blob/main/design/mvp/Async.md)
along with the [`stream`, `future`, and
`error-context`](WebAssembly/component-model#405) types.

By default, normal synchronous bindings are generated, but the user may opt-in
to async bindings for all or some of the imported and/or exported functions in
the target world and interfaces -- provided the default-enabled `async` feature
is enabled.

In addition, we generate `StreamPayload` and/or `FuturePayload` trait
implementations for any types appearing as the `T` in `stream<T>` or `future<T>`
in the WIT files, respectively.  That enables user code to call `new_stream` or
`new_future` to create `stream`s or `future`s with those payload types, then
write to them, read from them, and/or pass the readable end as a parameter to a
component import or return value of a component export.

Note that I've added new `core::abi::Instruction` enum variants to handle async
lifting and lowering, but they're currently tailored to the Rust generator and
will probably change somewhat as we add support for other languages.

This does not include any new tests; I'll add those in a follow-up commit.

Signed-off-by: Joel Dice <[email protected]>

This adds support for loading, compiling, linking, and running components which
use the [Async
ABI](https://github.com/WebAssembly/component-model/blob/main/design/mvp/Async.md)
along with the [`stream`, `future`, and
`error-context`](WebAssembly/component-model#405) types.
It also adds support for generating host bindings such that multiple host
functions can be run concurrently with guest tasks -- without monopolizing the
`Store`.

See the [implementation RFC](bytecodealliance/rfcs#38)
for details, as well as [this
repo](https://github.com/dicej/component-async-demo) containing end-to-end smoke
tests.

This is very much a work-in progress, with a number of tasks remaining:

- [ ] Avoid exposing global task IDs to guests and use per-instance IDs instead
- [ ] Track `task.return` type during compilation and assert the actual and expected types match at runtime
- [ ] Ensure all guest pointers are bounds-checked when lifting, lowering, or copying values
- [ ] Reduce code duplication in `wasmtime_cranelift::compiler::component`
- [ ] Add support for `(Typed)Func::call_concurrent` per the RFC
- [ ] Add support for multiplexing stream/future reads/writes and concurrent calls to guest exports per the RFC
- [ ] Refactor, clean up, and unify handling of backpressure, yields, and even polling
- [ ] Guard against reentrance where required (e.g. in certain fused adapter calls)
- [ ] Add integration test cases covering new functionality to tests/all/component_model (starting by porting over the tests in https://github.com/dicej/component-async-demo)
- [ ] Add binding generation test cases to crates/component-macro/tests
- [ ] Add WAST tests to tests/misc_testsuite/component-model
- [ ] Add support and test coverage for callback-less async functions (e.g. goroutines)
- [ ] Switch to back to upstream `wasm-tools` once bytecodealliance/wasm-tools#1895 has been merged and released

Signed-off-by: Joel Dice <[email protected]>

This adds support for loading, compiling, linking, and running components which
use the [Async
ABI](https://github.com/WebAssembly/component-model/blob/main/design/mvp/Async.md)
along with the [`stream`, `future`, and
`error-context`](WebAssembly/component-model#405) types.
It also adds support for generating host bindings such that multiple host
functions can be run concurrently with guest tasks -- without monopolizing the
`Store`.

See the [implementation RFC](bytecodealliance/rfcs#38)
for details, as well as [this
repo](https://github.com/dicej/component-async-demo) containing end-to-end smoke
tests.

This is very much a work-in progress, with a number of tasks remaining:

- [ ] Avoid exposing global task IDs to guests and use per-instance IDs instead
- [ ] Track `task.return` type during compilation and assert the actual and expected types match at runtime
- [ ] Ensure all guest pointers are bounds-checked when lifting, lowering, or copying values
- [ ] Reduce code duplication in `wasmtime_cranelift::compiler::component`
- [ ] Reduce code duplication between `StoreContextMut::on_fiber` and `concurrent::on_fiber`
- [ ] Minimize and/or document the use of unsafe code
- [ ] Add support for `(Typed)Func::call_concurrent` per the RFC
- [ ] Add support for multiplexing stream/future reads/writes and concurrent calls to guest exports per the RFC
- [ ] Refactor, clean up, and unify handling of backpressure, yields, and even polling
- [ ] Guard against reentrance where required (e.g. in certain fused adapter calls)
- [ ] Add integration test cases covering new functionality to tests/all/component_model (starting by porting over the tests in https://github.com/dicej/component-async-demo)
- [ ] Add binding generation test cases to crates/component-macro/tests
- [ ] Add WAST tests to tests/misc_testsuite/component-model
- [ ] Add support and test coverage for callback-less async functions (e.g. goroutines)
- [ ] Switch to back to upstream `wasm-tools` once bytecodealliance/wasm-tools#1895 has been merged and released

Signed-off-by: Joel Dice <[email protected]>

This adds support for encoding and parsing components which use the [Async
ABI](https://github.com/WebAssembly/component-model/blob/main/design/mvp/Async.md)
and associated canonical options and functions, along with the [`stream`,
`future`, and `error`](WebAssembly/component-model#405)
types.

Note that the `error` type was recently (about 30 minutes ago) renamed to
`error-context` in Luke's spec PR.  I haven't updated this implementation to
reflect that yet, but will do so in a follow-up commit.  That should allow us to
avoid conflicts with existing WIT files that use `error` as a type and/or
interface name.

This does not include any new tests; I'll also add those in a follow-up commit.

See bytecodealliance/rfcs#38 for more context.

Signed-off-by: Joel Dice <[email protected]>

add wasmparser::WasmFeatures support to wasm-compose

Signed-off-by: Joel Dice <[email protected]>

fix no-std build in readers.rs

Signed-off-by: Joel Dice <[email protected]>

rename `error` to `error-context` per latest spec

Signed-off-by: Joel Dice <[email protected]>

rename `error` to `error-context` per latest spec (part 2)

Also, parse string encoding and realloc from encoded `error-context.new` and
`error-context.debug-string` names.

Signed-off-by: Joel Dice <[email protected]>

add `wast` support for parsing async canon opts

And add tests/local/component-model-async/lift-async.wast for round-trip testing
of async lifts (more to come).

Signed-off-by: Joel Dice <[email protected]>

more wast async support and more tests

This also fixes a bug in `wasmprinter` keeping track of core functions.

Signed-off-by: Joel Dice <[email protected]>

more wast async support; add async tests; fix bugs

Signed-off-by: Joel Dice <[email protected]>

more component-model-async tests and fixes

Signed-off-by: Joel Dice <[email protected]>

add `wit-parser` tests for streams, futures, and error-contexts

Signed-off-by: Joel Dice <[email protected]>

add first `wit-component` async test

This required adding a new `wit_parser::decoding::decode_reader_with_features`
function for passing `WasmFeatures` to
`wasmparser::Validator::new_with_features`.

Signed-off-by: Joel Dice <[email protected]>

add more async tests

Signed-off-by: Joel Dice <[email protected]>

add `async-builtins` test for `wit-component`

Signed-off-by: Joel Dice <[email protected]>

add `async-streams-and-futures` test to `wit-component`

Signed-off-by: Joel Dice <[email protected]>

This adds support for generating bindings which use the [Async
ABI](https://github.com/WebAssembly/component-model/blob/main/design/mvp/Async.md)
along with the [`stream`, `future`, and
`error-context`](WebAssembly/component-model#405) types.

By default, normal synchronous bindings are generated, but the user may opt-in
to async bindings for all or some of the imported and/or exported functions in
the target world and interfaces -- provided the default-enabled `async` feature
is enabled.

In addition, we generate `StreamPayload` and/or `FuturePayload` trait
implementations for any types appearing as the `T` in `stream<T>` or `future<T>`
in the WIT files, respectively.  That enables user code to call `new_stream` or
`new_future` to create `stream`s or `future`s with those payload types, then
write to them, read from them, and/or pass the readable end as a parameter to a
component import or return value of a component export.

Note that I've added new `core::abi::Instruction` enum variants to handle async
lifting and lowering, but they're currently tailored to the Rust generator and
will probably change somewhat as we add support for other languages.

This does not include any new tests; I'll add those in a follow-up commit.

Signed-off-by: Joel Dice <[email protected]>

add `async: true` case to Rust `codegen_tests`

This ensures that all the codegen test WIT files produce compile-able bindings
with `async: true` (i.e. all imports lowered and all exports lifted using the
async ABI).  That revealed some issues involving resource methods and
constructors, as well as missing stub support, which I've resolved.

Signed-off-by: Joel Dice <[email protected]>

add codegen tests for futures, streams, and error-contexts

Signed-off-by: Joel Dice <[email protected]>

This adds support for generating bindings which use the [Async
ABI](https://github.com/WebAssembly/component-model/blob/main/design/mvp/Async.md)
along with the [`stream`, `future`, and
`error-context`](WebAssembly/component-model#405) types.

By default, normal synchronous bindings are generated, but the user may opt-in
to async bindings for all or some of the imported and/or exported functions in
the target world and interfaces -- provided the default-enabled `async` feature
is enabled.

In addition, we generate `StreamPayload` and/or `FuturePayload` trait
implementations for any types appearing as the `T` in `stream<T>` or `future<T>`
in the WIT files, respectively.  That enables user code to call `new_stream` or
`new_future` to create `stream`s or `future`s with those payload types, then
write to them, read from them, and/or pass the readable end as a parameter to a
component import or return value of a component export.

Note that I've added new `core::abi::Instruction` enum variants to handle async
lifting and lowering, but they're currently tailored to the Rust generator and
will probably change somewhat as we add support for other languages.

This does not include any new tests; I'll add those in a follow-up commit.

Signed-off-by: Joel Dice <[email protected]>

add `async: true` case to Rust `codegen_tests`

This ensures that all the codegen test WIT files produce compile-able bindings
with `async: true` (i.e. all imports lowered and all exports lifted using the
async ABI).  That revealed some issues involving resource methods and
constructors, as well as missing stub support, which I've resolved.

Signed-off-by: Joel Dice <[email protected]>

add codegen tests for futures, streams, and error-contexts

Signed-off-by: Joel Dice <[email protected]>

This adds support for loading, compiling, linking, and running components which
use the [Async
ABI](https://github.com/WebAssembly/component-model/blob/main/design/mvp/Async.md)
along with the [`stream`, `future`, and
`error-context`](WebAssembly/component-model#405) types.
It also adds support for generating host bindings such that multiple host
functions can be run concurrently with guest tasks -- without monopolizing the
`Store`.

See the [implementation RFC](bytecodealliance/rfcs#38)
for details, as well as [this
repo](https://github.com/dicej/component-async-demo) containing end-to-end smoke
tests.

This is very much a work-in progress, with a number of tasks remaining:

- [ ] Avoid exposing global task IDs to guests and use per-instance IDs instead
- [ ] Track `task.return` type during compilation and assert the actual and expected types match at runtime
- [ ] Ensure all guest pointers are bounds-checked when lifting, lowering, or copying values
- [ ] Reduce code duplication in `wasmtime_cranelift::compiler::component`
- [ ] Reduce code duplication between `StoreContextMut::on_fiber` and `concurrent::on_fiber`
- [ ] Minimize and/or document the use of unsafe code
- [ ] Add support for `(Typed)Func::call_concurrent` per the RFC
- [ ] Add support for multiplexing stream/future reads/writes and concurrent calls to guest exports per the RFC
- [ ] Refactor, clean up, and unify handling of backpressure, yields, and even polling
- [ ] Guard against reentrance where required (e.g. in certain fused adapter calls)
- [ ] Add integration test cases covering new functionality to tests/all/component_model (starting by porting over the tests in https://github.com/dicej/component-async-demo)
- [ ] Add binding generation test cases to crates/component-macro/tests
- [ ] Add WAST tests to tests/misc_testsuite/component-model
- [ ] Add support and test coverage for callback-less async functions (e.g. goroutines)
- [ ] Switch to back to upstream `wasm-tools` once bytecodealliance/wasm-tools#1895 has been merged and released

Signed-off-by: Joel Dice <[email protected]>

fix clippy warnings and bench/fuzzing errors

Signed-off-by: Joel Dice <[email protected]>

revert atomic.wit whitespace change

Signed-off-by: Joel Dice <[email protected]>

fix build when component-model disabled

Signed-off-by: Joel Dice <[email protected]>

bless component-macro expected output

Signed-off-by: Joel Dice <[email protected]>

fix no-std build error

Signed-off-by: Joel Dice <[email protected]>

fix build with --no-default-features --features runtime,component-model

Signed-off-by: Joel Dice <[email protected]>

partly fix no-std build

It's still broken due to the use of `std::collections::HashMap` in
crates/wasmtime/src/runtime/vm/component.rs.  I'll address that as part of the
work to avoid exposing global task/future/stream/error-context handles to
guests.

Signed-off-by: Joel Dice <[email protected]>

maintain per-instance tables for futures, streams, and error-contexts

Signed-off-by: Joel Dice <[email protected]>

This adds support for encoding and parsing components which use the [Async
ABI](https://github.com/WebAssembly/component-model/blob/main/design/mvp/Async.md)
and associated canonical options and functions, along with the [`stream`,
`future`, and `error`](WebAssembly/component-model#405)
types.

Note that the `error` type was recently (about 30 minutes ago) renamed to
`error-context` in Luke's spec PR.  I haven't updated this implementation to
reflect that yet, but will do so in a follow-up commit.  That should allow us to
avoid conflicts with existing WIT files that use `error` as a type and/or
interface name.

This does not include any new tests; I'll also add those in a follow-up commit.

See bytecodealliance/rfcs#38 for more context.

Signed-off-by: Joel Dice <[email protected]>

add wasmparser::WasmFeatures support to wasm-compose

Signed-off-by: Joel Dice <[email protected]>

fix no-std build in readers.rs

Signed-off-by: Joel Dice <[email protected]>

rename `error` to `error-context` per latest spec

Signed-off-by: Joel Dice <[email protected]>

rename `error` to `error-context` per latest spec (part 2)

Also, parse string encoding and realloc from encoded `error-context.new` and
`error-context.debug-string` names.

Signed-off-by: Joel Dice <[email protected]>

add `wast` support for parsing async canon opts

And add tests/local/component-model-async/lift-async.wast for round-trip testing
of async lifts (more to come).

Signed-off-by: Joel Dice <[email protected]>

more wast async support and more tests

This also fixes a bug in `wasmprinter` keeping track of core functions.

Signed-off-by: Joel Dice <[email protected]>

more wast async support; add async tests; fix bugs

Signed-off-by: Joel Dice <[email protected]>

more component-model-async tests and fixes

Signed-off-by: Joel Dice <[email protected]>

add `wit-parser` tests for streams, futures, and error-contexts

Signed-off-by: Joel Dice <[email protected]>

add first `wit-component` async test

This required adding a new `wit_parser::decoding::decode_reader_with_features`
function for passing `WasmFeatures` to
`wasmparser::Validator::new_with_features`.

Signed-off-by: Joel Dice <[email protected]>

add more async tests

Signed-off-by: Joel Dice <[email protected]>

add `async-builtins` test for `wit-component`

Signed-off-by: Joel Dice <[email protected]>

add `async-streams-and-futures` test to `wit-component`

Signed-off-by: Joel Dice <[email protected]>

This adds support for loading, compiling, linking, and running components which
use the [Async
ABI](https://github.com/WebAssembly/component-model/blob/main/design/mvp/Async.md)
along with the [`stream`, `future`, and
`error-context`](WebAssembly/component-model#405) types.
It also adds support for generating host bindings such that multiple host
functions can be run concurrently with guest tasks -- without monopolizing the
`Store`.

See the [implementation RFC](bytecodealliance/rfcs#38)
for details, as well as [this
repo](https://github.com/dicej/component-async-demo) containing end-to-end smoke
tests.

This is very much a work-in progress, with a number of tasks remaining:

- [ ] Avoid exposing global task IDs to guests and use per-instance IDs instead
- [ ] Track `task.return` type during compilation and assert the actual and expected types match at runtime
- [ ] Ensure all guest pointers are bounds-checked when lifting, lowering, or copying values
- [ ] Reduce code duplication in `wasmtime_cranelift::compiler::component`
- [ ] Reduce code duplication between `StoreContextMut::on_fiber` and `concurrent::on_fiber`
- [ ] Minimize and/or document the use of unsafe code
- [ ] Add support for `(Typed)Func::call_concurrent` per the RFC
- [ ] Add support for multiplexing stream/future reads/writes and concurrent calls to guest exports per the RFC
- [ ] Refactor, clean up, and unify handling of backpressure, yields, and even polling
- [ ] Guard against reentrance where required (e.g. in certain fused adapter calls)
- [ ] Add integration test cases covering new functionality to tests/all/component_model (starting by porting over the tests in https://github.com/dicej/component-async-demo)
- [ ] Add binding generation test cases to crates/component-macro/tests
- [ ] Add WAST tests to tests/misc_testsuite/component-model
- [ ] Add support and test coverage for callback-less async functions (e.g. goroutines)
- [ ] Switch to back to upstream `wasm-tools` once bytecodealliance/wasm-tools#1895 has been merged and released

Signed-off-by: Joel Dice <[email protected]>

fix clippy warnings and bench/fuzzing errors

Signed-off-by: Joel Dice <[email protected]>

revert atomic.wit whitespace change

Signed-off-by: Joel Dice <[email protected]>

fix build when component-model disabled

Signed-off-by: Joel Dice <[email protected]>

bless component-macro expected output

Signed-off-by: Joel Dice <[email protected]>

fix no-std build error

Signed-off-by: Joel Dice <[email protected]>

fix build with --no-default-features --features runtime,component-model

Signed-off-by: Joel Dice <[email protected]>

partly fix no-std build

It's still broken due to the use of `std::collections::HashMap` in
crates/wasmtime/src/runtime/vm/component.rs.  I'll address that as part of the
work to avoid exposing global task/future/stream/error-context handles to
guests.

Signed-off-by: Joel Dice <[email protected]>

maintain per-instance tables for futures, streams, and error-contexts

Signed-off-by: Joel Dice <[email protected]>

refactor task/stream/future handle lifting/lowering

This addresses a couple of issues:

- Previously, we were passing task/stream/future/error-context reps directly to
  instances while keeping track of which instance had access to which rep.  That
  worked fine in that there was no way to forge access to inaccessible reps, but
  it leaked information about what other instances were doing.  Now we maintain
  per-instance waitable and error-context tables which map the reps to and from
  the handles which the instance sees.

- The `no_std` build was broken due to use of `HashMap` in
  `runtime::vm::component`, which is now fixed.

Note that we use one single table per instance for all tasks, streams, and
futures.  This is partly necessary because, when async events are delivered to
the guest, it wouldn't have enough context to know which stream or future we're
talking about if each unique stream and future type had its own table.  So at
minimum, we need to use the same table for all streams (regardless of payload
type), and likewise for futures.  Also, per
WebAssembly/component-model#395 (comment),
the plan is to move towards a shared table for all resource types as well, so
this moves us in that direction.

Signed-off-by: Joel Dice <[email protected]>

fix wave breakage due to new stream/future/error-context types

Signed-off-by: Joel Dice <[email protected]>

This adds support for encoding and parsing components which use the [Async
ABI](https://github.com/WebAssembly/component-model/blob/main/design/mvp/Async.md)
and associated canonical options and functions, along with the [`stream`,
`future`, and `error`](WebAssembly/component-model#405)
types.

See bytecodealliance/rfcs#38 for more context.

Signed-off-by: Joel Dice <[email protected]>

add wasmparser::WasmFeatures support to wasm-compose

Signed-off-by: Joel Dice <[email protected]>

fix no-std build in readers.rs

Signed-off-by: Joel Dice <[email protected]>

rename `error` to `error-context` per latest spec

Signed-off-by: Joel Dice <[email protected]>

rename `error` to `error-context` per latest spec (part 2)

Also, parse string encoding and realloc from encoded `error-context.new` and
`error-context.debug-string` names.

Signed-off-by: Joel Dice <[email protected]>

add `wast` support for parsing async canon opts

And add tests/local/component-model-async/lift-async.wast for round-trip testing
of async lifts (more to come).

Signed-off-by: Joel Dice <[email protected]>

more wast async support and more tests

This also fixes a bug in `wasmprinter` keeping track of core functions.

Signed-off-by: Joel Dice <[email protected]>

more wast async support; add async tests; fix bugs

Signed-off-by: Joel Dice <[email protected]>

more component-model-async tests and fixes

Signed-off-by: Joel Dice <[email protected]>

add `wit-parser` tests for streams, futures, and error-contexts

Signed-off-by: Joel Dice <[email protected]>

add first `wit-component` async test

This required adding a new `wit_parser::decoding::decode_reader_with_features`
function for passing `WasmFeatures` to
`wasmparser::Validator::new_with_features`.

Signed-off-by: Joel Dice <[email protected]>

add more async tests

Signed-off-by: Joel Dice <[email protected]>

add `async-builtins` test for `wit-component`

Signed-off-by: Joel Dice <[email protected]>

add `async-streams-and-futures` test to `wit-component`

Signed-off-by: Joel Dice <[email protected]>

fix stream/future type handling for exported interfaces

Signed-off-by: Joel Dice <[email protected]>

support callback-less (AKA stackful) async lifts

Signed-off-by: Joel Dice <[email protected]>

This adds support for generating bindings which use the [Async
ABI](https://github.com/WebAssembly/component-model/blob/main/design/mvp/Async.md)
along with the [`stream`, `future`, and
`error-context`](WebAssembly/component-model#405) types.

By default, normal synchronous bindings are generated, but the user may opt-in
to async bindings for all or some of the imported and/or exported functions in
the target world and interfaces -- provided the default-enabled `async` feature
is enabled.

In addition, we generate `StreamPayload` and/or `FuturePayload` trait
implementations for any types appearing as the `T` in `stream<T>` or `future<T>`
in the WIT files, respectively.  That enables user code to call `new_stream` or
`new_future` to create `stream`s or `future`s with those payload types, then
write to them, read from them, and/or pass the readable end as a parameter to a
component import or return value of a component export.

Note that I've added new `core::abi::Instruction` enum variants to handle async
lifting and lowering, but they're currently tailored to the Rust generator and
will probably change somewhat as we add support for other languages.

This does not include any new tests; I'll add those in a follow-up commit.

Signed-off-by: Joel Dice <[email protected]>

add `async: true` case to Rust `codegen_tests`

This ensures that all the codegen test WIT files produce compile-able bindings
with `async: true` (i.e. all imports lowered and all exports lifted using the
async ABI).  That revealed some issues involving resource methods and
constructors, as well as missing stub support, which I've resolved.

Signed-off-by: Joel Dice <[email protected]>

add codegen tests for futures, streams, and error-contexts

Signed-off-by: Joel Dice <[email protected]>

remove async_support::poll_future

It was both unsafe to use and intended only for testing (and not even good for
that, it turns out).

Signed-off-by: Joel Dice <[email protected]>

add stream/future read/write cancellation support

Also, fix some issues with stream/future payload lifting/lowering which I
_thought_ I had already tested but actually hadn't.

Signed-off-by: Joel Dice <[email protected]>

support callback-less (AKA stackful) async lifts

Signed-off-by: Joel Dice <[email protected]>

This adds support for loading, compiling, linking, and running components which
use the [Async
ABI](https://github.com/WebAssembly/component-model/blob/main/design/mvp/Async.md)
along with the [`stream`, `future`, and
`error-context`](WebAssembly/component-model#405) types.
It also adds support for generating host bindings such that multiple host
functions can be run concurrently with guest tasks -- without monopolizing the
`Store`.

See the [implementation RFC](bytecodealliance/rfcs#38)
for details, as well as [this
repo](https://github.com/dicej/component-async-demo) containing end-to-end smoke
tests.

Signed-off-by: Joel Dice <[email protected]>

fix clippy warnings and bench/fuzzing errors

Signed-off-by: Joel Dice <[email protected]>

revert atomic.wit whitespace change

Signed-off-by: Joel Dice <[email protected]>

fix build when component-model disabled

Signed-off-by: Joel Dice <[email protected]>

bless component-macro expected output

Signed-off-by: Joel Dice <[email protected]>

fix no-std build error

Signed-off-by: Joel Dice <[email protected]>

fix build with --no-default-features --features runtime,component-model

Signed-off-by: Joel Dice <[email protected]>

partly fix no-std build

It's still broken due to the use of `std::collections::HashMap` in
crates/wasmtime/src/runtime/vm/component.rs.  I'll address that as part of the
work to avoid exposing global task/future/stream/error-context handles to
guests.

Signed-off-by: Joel Dice <[email protected]>

maintain per-instance tables for futures, streams, and error-contexts

Signed-off-by: Joel Dice <[email protected]>

refactor task/stream/future handle lifting/lowering

This addresses a couple of issues:

- Previously, we were passing task/stream/future/error-context reps directly to
  instances while keeping track of which instance had access to which rep.  That
  worked fine in that there was no way to forge access to inaccessible reps, but
  it leaked information about what other instances were doing.  Now we maintain
  per-instance waitable and error-context tables which map the reps to and from
  the handles which the instance sees.

- The `no_std` build was broken due to use of `HashMap` in
  `runtime::vm::component`, which is now fixed.

Note that we use one single table per instance for all tasks, streams, and
futures.  This is partly necessary because, when async events are delivered to
the guest, it wouldn't have enough context to know which stream or future we're
talking about if each unique stream and future type had its own table.  So at
minimum, we need to use the same table for all streams (regardless of payload
type), and likewise for futures.  Also, per
WebAssembly/component-model#395 (comment),
the plan is to move towards a shared table for all resource types as well, so
this moves us in that direction.

Signed-off-by: Joel Dice <[email protected]>

fix wave breakage due to new stream/future/error-context types

Signed-off-by: Joel Dice <[email protected]>

switch wasm-tools to v1.220.0-based branch

Signed-off-by: Joel Dice <[email protected]>

check `task.return` type at runtime

We can't statically verify a given call to `task.return` corresponds to the
expected core signature appropriate for the currently running task, so we must
do so at runtime.  In order to make that check efficient, we intern the types.

My initial plan was to use `ModuleInternedTypeIndex` and/or `VMSharedTypeIndex`
for interning, but that got hairy with WasmGC considerations, so instead I added
new fields to `ComponentTypes` and `ComponentTypesBuilder`.

Signed-off-by: Joel Dice <[email protected]>

add `TypedFunc::call_concurrent` and refine stream/future APIs

This implements what I proposed in https://github.com/dicej/rfcs/blob/component-async/accepted/component-model-async.md#wasmtime.  Specifically, it adds:

- A new `Promise` type, useful for working with concurrent operations that require access to a `Store` to make progress.
- A new `PromisesUnordered` type for `await`ing multiple promises concurrently
-`TypedFunc::call_concurrent` (which returns a `Promise`), allowing multiple host->guest calls to run concurrently on the same instance.
- Updated `{Stream|Future}{Writer|Reader}` APIs which use `Promise`

The upshot is that the embedder can now ergonomically manage arbitrary numbers
of concurrent operations.  Previously, this was a lot more difficult to do
without accidentally starving some of the operations due to another one
monopolizing the `Store`.

Finally, this includes various refactorings and fixes for bugs exposed by the
newer, more versatile APIs.

Signed-off-by: Joel Dice <[email protected]>

clean up verbosity in component/func.rs

Signed-off-by: Joel Dice <[email protected]>

snapshot

Signed-off-by: Joel Dice <[email protected]>

implement stream/future read/write cancellation

This required a somewhat viral addition of `Send` and `Sync` bounds for async
host function closure types, unfortunately.

Signed-off-by: Joel Dice <[email protected]>

add `Func::call_concurrent` and `LinkerInstance::func_new_concurrent`

Signed-off-by: Joel Dice <[email protected]>

dynamic API support for streams/futures/error-contexts

Signed-off-by: Joel Dice <[email protected]>

support callback-less (AKA stackful) async lifts

Signed-off-by: Joel Dice <[email protected]>

This adds support for loading, compiling, linking, and running components which
use the [Async
ABI](https://github.com/WebAssembly/component-model/blob/main/design/mvp/Async.md)
along with the [`stream`, `future`, and
`error-context`](WebAssembly/component-model#405) types.
It also adds support for generating host bindings such that multiple host
functions can be run concurrently with guest tasks -- without monopolizing the
`Store`.

See the [implementation RFC](bytecodealliance/rfcs#38)
for details, as well as [this
repo](https://github.com/dicej/component-async-demo) containing end-to-end smoke
tests.

Signed-off-by: Joel Dice <[email protected]>

fix clippy warnings and bench/fuzzing errors

Signed-off-by: Joel Dice <[email protected]>

revert atomic.wit whitespace change

Signed-off-by: Joel Dice <[email protected]>

fix build when component-model disabled

Signed-off-by: Joel Dice <[email protected]>

bless component-macro expected output

Signed-off-by: Joel Dice <[email protected]>

fix no-std build error

Signed-off-by: Joel Dice <[email protected]>

fix build with --no-default-features --features runtime,component-model

Signed-off-by: Joel Dice <[email protected]>

partly fix no-std build

It's still broken due to the use of `std::collections::HashMap` in
crates/wasmtime/src/runtime/vm/component.rs.  I'll address that as part of the
work to avoid exposing global task/future/stream/error-context handles to
guests.

Signed-off-by: Joel Dice <[email protected]>

maintain per-instance tables for futures, streams, and error-contexts

Signed-off-by: Joel Dice <[email protected]>

refactor task/stream/future handle lifting/lowering

This addresses a couple of issues:

- Previously, we were passing task/stream/future/error-context reps directly to
  instances while keeping track of which instance had access to which rep.  That
  worked fine in that there was no way to forge access to inaccessible reps, but
  it leaked information about what other instances were doing.  Now we maintain
  per-instance waitable and error-context tables which map the reps to and from
  the handles which the instance sees.

- The `no_std` build was broken due to use of `HashMap` in
  `runtime::vm::component`, which is now fixed.

Note that we use one single table per instance for all tasks, streams, and
futures.  This is partly necessary because, when async events are delivered to
the guest, it wouldn't have enough context to know which stream or future we're
talking about if each unique stream and future type had its own table.  So at
minimum, we need to use the same table for all streams (regardless of payload
type), and likewise for futures.  Also, per
WebAssembly/component-model#395 (comment),
the plan is to move towards a shared table for all resource types as well, so
this moves us in that direction.

Signed-off-by: Joel Dice <[email protected]>

fix wave breakage due to new stream/future/error-context types

Signed-off-by: Joel Dice <[email protected]>

switch wasm-tools to v1.220.0-based branch

Signed-off-by: Joel Dice <[email protected]>

check `task.return` type at runtime

We can't statically verify a given call to `task.return` corresponds to the
expected core signature appropriate for the currently running task, so we must
do so at runtime.  In order to make that check efficient, we intern the types.

My initial plan was to use `ModuleInternedTypeIndex` and/or `VMSharedTypeIndex`
for interning, but that got hairy with WasmGC considerations, so instead I added
new fields to `ComponentTypes` and `ComponentTypesBuilder`.

Signed-off-by: Joel Dice <[email protected]>

add `TypedFunc::call_concurrent` and refine stream/future APIs

This implements what I proposed in https://github.com/dicej/rfcs/blob/component-async/accepted/component-model-async.md#wasmtime.  Specifically, it adds:

- A new `Promise` type, useful for working with concurrent operations that require access to a `Store` to make progress.
- A new `PromisesUnordered` type for `await`ing multiple promises concurrently
-`TypedFunc::call_concurrent` (which returns a `Promise`), allowing multiple host->guest calls to run concurrently on the same instance.
- Updated `{Stream|Future}{Writer|Reader}` APIs which use `Promise`

The upshot is that the embedder can now ergonomically manage arbitrary numbers
of concurrent operations.  Previously, this was a lot more difficult to do
without accidentally starving some of the operations due to another one
monopolizing the `Store`.

Finally, this includes various refactorings and fixes for bugs exposed by the
newer, more versatile APIs.

Signed-off-by: Joel Dice <[email protected]>

clean up verbosity in component/func.rs

Signed-off-by: Joel Dice <[email protected]>

snapshot

Signed-off-by: Joel Dice <[email protected]>

implement stream/future read/write cancellation

This required a somewhat viral addition of `Send` and `Sync` bounds for async
host function closure types, unfortunately.

Signed-off-by: Joel Dice <[email protected]>

add `Func::call_concurrent` and `LinkerInstance::func_new_concurrent`

Signed-off-by: Joel Dice <[email protected]>

dynamic API support for streams/futures/error-contexts

Signed-off-by: Joel Dice <[email protected]>

support callback-less (AKA stackful) async lifts

Signed-off-by: Joel Dice <[email protected]>

This adds support for loading, compiling, linking, and running components which
use the [Async
ABI](https://github.com/WebAssembly/component-model/blob/main/design/mvp/Async.md)
along with the [`stream`, `future`, and
`error-context`](WebAssembly/component-model#405) types.
It also adds support for generating host bindings such that multiple host
functions can be run concurrently with guest tasks -- without monopolizing the
`Store`.

See the [implementation RFC](bytecodealliance/rfcs#38)
for details, as well as [this
repo](https://github.com/dicej/component-async-demo) containing end-to-end smoke
tests.

Signed-off-by: Joel Dice <[email protected]>

fix clippy warnings and bench/fuzzing errors

Signed-off-by: Joel Dice <[email protected]>

revert atomic.wit whitespace change

Signed-off-by: Joel Dice <[email protected]>

fix build when component-model disabled

Signed-off-by: Joel Dice <[email protected]>

bless component-macro expected output

Signed-off-by: Joel Dice <[email protected]>

fix no-std build error

Signed-off-by: Joel Dice <[email protected]>

fix build with --no-default-features --features runtime,component-model

Signed-off-by: Joel Dice <[email protected]>

partly fix no-std build

It's still broken due to the use of `std::collections::HashMap` in
crates/wasmtime/src/runtime/vm/component.rs.  I'll address that as part of the
work to avoid exposing global task/future/stream/error-context handles to
guests.

Signed-off-by: Joel Dice <[email protected]>

maintain per-instance tables for futures, streams, and error-contexts

Signed-off-by: Joel Dice <[email protected]>

refactor task/stream/future handle lifting/lowering

This addresses a couple of issues:

- Previously, we were passing task/stream/future/error-context reps directly to
  instances while keeping track of which instance had access to which rep.  That
  worked fine in that there was no way to forge access to inaccessible reps, but
  it leaked information about what other instances were doing.  Now we maintain
  per-instance waitable and error-context tables which map the reps to and from
  the handles which the instance sees.

- The `no_std` build was broken due to use of `HashMap` in
  `runtime::vm::component`, which is now fixed.

Note that we use one single table per instance for all tasks, streams, and
futures.  This is partly necessary because, when async events are delivered to
the guest, it wouldn't have enough context to know which stream or future we're
talking about if each unique stream and future type had its own table.  So at
minimum, we need to use the same table for all streams (regardless of payload
type), and likewise for futures.  Also, per
WebAssembly/component-model#395 (comment),
the plan is to move towards a shared table for all resource types as well, so
this moves us in that direction.

Signed-off-by: Joel Dice <[email protected]>

fix wave breakage due to new stream/future/error-context types

Signed-off-by: Joel Dice <[email protected]>

switch wasm-tools to v1.220.0-based branch

Signed-off-by: Joel Dice <[email protected]>

check `task.return` type at runtime

We can't statically verify a given call to `task.return` corresponds to the
expected core signature appropriate for the currently running task, so we must
do so at runtime.  In order to make that check efficient, we intern the types.

My initial plan was to use `ModuleInternedTypeIndex` and/or `VMSharedTypeIndex`
for interning, but that got hairy with WasmGC considerations, so instead I added
new fields to `ComponentTypes` and `ComponentTypesBuilder`.

Signed-off-by: Joel Dice <[email protected]>

add `TypedFunc::call_concurrent` and refine stream/future APIs

This implements what I proposed in https://github.com/dicej/rfcs/blob/component-async/accepted/component-model-async.md#wasmtime.  Specifically, it adds:

- A new `Promise` type, useful for working with concurrent operations that require access to a `Store` to make progress.
- A new `PromisesUnordered` type for `await`ing multiple promises concurrently
-`TypedFunc::call_concurrent` (which returns a `Promise`), allowing multiple host->guest calls to run concurrently on the same instance.
- Updated `{Stream|Future}{Writer|Reader}` APIs which use `Promise`

The upshot is that the embedder can now ergonomically manage arbitrary numbers
of concurrent operations.  Previously, this was a lot more difficult to do
without accidentally starving some of the operations due to another one
monopolizing the `Store`.

Finally, this includes various refactorings and fixes for bugs exposed by the
newer, more versatile APIs.

Signed-off-by: Joel Dice <[email protected]>

clean up verbosity in component/func.rs

Signed-off-by: Joel Dice <[email protected]>

snapshot

Signed-off-by: Joel Dice <[email protected]>

implement stream/future read/write cancellation

This required a somewhat viral addition of `Send` and `Sync` bounds for async
host function closure types, unfortunately.

Signed-off-by: Joel Dice <[email protected]>

add `Func::call_concurrent` and `LinkerInstance::func_new_concurrent`

Signed-off-by: Joel Dice <[email protected]>

dynamic API support for streams/futures/error-contexts

Signed-off-by: Joel Dice <[email protected]>

support callback-less (AKA stackful) async lifts

Signed-off-by: Joel Dice <[email protected]>

This adds support for encoding and parsing components which use the [Async
ABI](https://github.com/WebAssembly/component-model/blob/main/design/mvp/Async.md)
and associated canonical options and functions, along with the [`stream`,
`future`, and `error`](WebAssembly/component-model#405)
types.

See bytecodealliance/rfcs#38 for more context.

Signed-off-by: Joel Dice <[email protected]>

add wasmparser::WasmFeatures support to wasm-compose

Signed-off-by: Joel Dice <[email protected]>

fix no-std build in readers.rs

Signed-off-by: Joel Dice <[email protected]>

rename `error` to `error-context` per latest spec

Signed-off-by: Joel Dice <[email protected]>

rename `error` to `error-context` per latest spec (part 2)

Also, parse string encoding and realloc from encoded `error-context.new` and
`error-context.debug-string` names.

Signed-off-by: Joel Dice <[email protected]>

add `wast` support for parsing async canon opts

And add tests/local/component-model-async/lift-async.wast for round-trip testing
of async lifts (more to come).

Signed-off-by: Joel Dice <[email protected]>

more wast async support and more tests

This also fixes a bug in `wasmprinter` keeping track of core functions.

Signed-off-by: Joel Dice <[email protected]>

more wast async support; add async tests; fix bugs

Signed-off-by: Joel Dice <[email protected]>

more component-model-async tests and fixes

Signed-off-by: Joel Dice <[email protected]>

add `wit-parser` tests for streams, futures, and error-contexts

Signed-off-by: Joel Dice <[email protected]>

add first `wit-component` async test

This required adding a new `wit_parser::decoding::decode_reader_with_features`
function for passing `WasmFeatures` to
`wasmparser::Validator::new_with_features`.

Signed-off-by: Joel Dice <[email protected]>

add more async tests

Signed-off-by: Joel Dice <[email protected]>

add `async-builtins` test for `wit-component`

Signed-off-by: Joel Dice <[email protected]>

add `async-streams-and-futures` test to `wit-component`

Signed-off-by: Joel Dice <[email protected]>

fix stream/future type handling for exported interfaces

Signed-off-by: Joel Dice <[email protected]>

support callback-less (AKA stackful) async lifts

Signed-off-by: Joel Dice <[email protected]>

This adds support for loading, compiling, linking, and running components which
use the [Async
ABI](https://github.com/WebAssembly/component-model/blob/main/design/mvp/Async.md)
along with the [`stream`, `future`, and
`error-context`](WebAssembly/component-model#405) types.
It also adds support for generating host bindings such that multiple host
functions can be run concurrently with guest tasks -- without monopolizing the
`Store`.

See the [implementation RFC](bytecodealliance/rfcs#38)
for details, as well as [this
repo](https://github.com/dicej/component-async-demo) containing end-to-end smoke
tests.

Signed-off-by: Joel Dice <[email protected]>

fix clippy warnings and bench/fuzzing errors

Signed-off-by: Joel Dice <[email protected]>

revert atomic.wit whitespace change

Signed-off-by: Joel Dice <[email protected]>

fix build when component-model disabled

Signed-off-by: Joel Dice <[email protected]>

bless component-macro expected output

Signed-off-by: Joel Dice <[email protected]>

fix no-std build error

Signed-off-by: Joel Dice <[email protected]>

fix build with --no-default-features --features runtime,component-model

Signed-off-by: Joel Dice <[email protected]>

partly fix no-std build

It's still broken due to the use of `std::collections::HashMap` in
crates/wasmtime/src/runtime/vm/component.rs.  I'll address that as part of the
work to avoid exposing global task/future/stream/error-context handles to
guests.

Signed-off-by: Joel Dice <[email protected]>

maintain per-instance tables for futures, streams, and error-contexts

Signed-off-by: Joel Dice <[email protected]>

refactor task/stream/future handle lifting/lowering

This addresses a couple of issues:

- Previously, we were passing task/stream/future/error-context reps directly to
  instances while keeping track of which instance had access to which rep.  That
  worked fine in that there was no way to forge access to inaccessible reps, but
  it leaked information about what other instances were doing.  Now we maintain
  per-instance waitable and error-context tables which map the reps to and from
  the handles which the instance sees.

- The `no_std` build was broken due to use of `HashMap` in
  `runtime::vm::component`, which is now fixed.

Note that we use one single table per instance for all tasks, streams, and
futures.  This is partly necessary because, when async events are delivered to
the guest, it wouldn't have enough context to know which stream or future we're
talking about if each unique stream and future type had its own table.  So at
minimum, we need to use the same table for all streams (regardless of payload
type), and likewise for futures.  Also, per
WebAssembly/component-model#395 (comment),
the plan is to move towards a shared table for all resource types as well, so
this moves us in that direction.

Signed-off-by: Joel Dice <[email protected]>

fix wave breakage due to new stream/future/error-context types

Signed-off-by: Joel Dice <[email protected]>

switch wasm-tools to v1.220.0-based branch

Signed-off-by: Joel Dice <[email protected]>

check `task.return` type at runtime

We can't statically verify a given call to `task.return` corresponds to the
expected core signature appropriate for the currently running task, so we must
do so at runtime.  In order to make that check efficient, we intern the types.

My initial plan was to use `ModuleInternedTypeIndex` and/or `VMSharedTypeIndex`
for interning, but that got hairy with WasmGC considerations, so instead I added
new fields to `ComponentTypes` and `ComponentTypesBuilder`.

Signed-off-by: Joel Dice <[email protected]>

add `TypedFunc::call_concurrent` and refine stream/future APIs

This implements what I proposed in https://github.com/dicej/rfcs/blob/component-async/accepted/component-model-async.md#wasmtime.  Specifically, it adds:

- A new `Promise` type, useful for working with concurrent operations that require access to a `Store` to make progress.
- A new `PromisesUnordered` type for `await`ing multiple promises concurrently
-`TypedFunc::call_concurrent` (which returns a `Promise`), allowing multiple host->guest calls to run concurrently on the same instance.
- Updated `{Stream|Future}{Writer|Reader}` APIs which use `Promise`

The upshot is that the embedder can now ergonomically manage arbitrary numbers
of concurrent operations.  Previously, this was a lot more difficult to do
without accidentally starving some of the operations due to another one
monopolizing the `Store`.

Finally, this includes various refactorings and fixes for bugs exposed by the
newer, more versatile APIs.

Signed-off-by: Joel Dice <[email protected]>

clean up verbosity in component/func.rs

Signed-off-by: Joel Dice <[email protected]>

snapshot

Signed-off-by: Joel Dice <[email protected]>

implement stream/future read/write cancellation

This required a somewhat viral addition of `Send` and `Sync` bounds for async
host function closure types, unfortunately.

Signed-off-by: Joel Dice <[email protected]>

add `Func::call_concurrent` and `LinkerInstance::func_new_concurrent`

Signed-off-by: Joel Dice <[email protected]>

dynamic API support for streams/futures/error-contexts

Signed-off-by: Joel Dice <[email protected]>

support callback-less (AKA stackful) async lifts

Signed-off-by: Joel Dice <[email protected]>

This adds support for encoding and parsing components which use the [Async
ABI](https://github.com/WebAssembly/component-model/blob/main/design/mvp/Async.md)
and associated canonical options and functions, along with the [`stream`,
`future`, and `error`](WebAssembly/component-model#405)
types.

See bytecodealliance/rfcs#38 for more context.

Signed-off-by: Joel Dice <[email protected]>

add wasmparser::WasmFeatures support to wasm-compose

Signed-off-by: Joel Dice <[email protected]>

fix no-std build in readers.rs

Signed-off-by: Joel Dice <[email protected]>

rename `error` to `error-context` per latest spec

Signed-off-by: Joel Dice <[email protected]>

rename `error` to `error-context` per latest spec (part 2)

Also, parse string encoding and realloc from encoded `error-context.new` and
`error-context.debug-string` names.

Signed-off-by: Joel Dice <[email protected]>

add `wast` support for parsing async canon opts

And add tests/local/component-model-async/lift-async.wast for round-trip testing
of async lifts (more to come).

Signed-off-by: Joel Dice <[email protected]>

more wast async support and more tests

This also fixes a bug in `wasmprinter` keeping track of core functions.

Signed-off-by: Joel Dice <[email protected]>

more wast async support; add async tests; fix bugs

Signed-off-by: Joel Dice <[email protected]>

more component-model-async tests and fixes

Signed-off-by: Joel Dice <[email protected]>

add `wit-parser` tests for streams, futures, and error-contexts

Signed-off-by: Joel Dice <[email protected]>

add first `wit-component` async test

This required adding a new `wit_parser::decoding::decode_reader_with_features`
function for passing `WasmFeatures` to
`wasmparser::Validator::new_with_features`.

Signed-off-by: Joel Dice <[email protected]>

add more async tests

Signed-off-by: Joel Dice <[email protected]>

add `async-builtins` test for `wit-component`

Signed-off-by: Joel Dice <[email protected]>

add `async-streams-and-futures` test to `wit-component`

Signed-off-by: Joel Dice <[email protected]>

fix stream/future type handling for exported interfaces

Signed-off-by: Joel Dice <[email protected]>

support callback-less (AKA stackful) async lifts

Signed-off-by: Joel Dice <[email protected]>

address review feedback

Signed-off-by: Joel Dice <[email protected]>

* Add support for async ABI, futures, streams, and errors

This adds support for encoding and parsing components which use the [Async
ABI](https://github.com/WebAssembly/component-model/blob/main/design/mvp/Async.md)
and associated canonical options and functions, along with the [`stream`,
`future`, and `error`](WebAssembly/component-model#405)
types.

See bytecodealliance/rfcs#38 for more context.

Signed-off-by: Joel Dice <[email protected]>

add wasmparser::WasmFeatures support to wasm-compose

Signed-off-by: Joel Dice <[email protected]>

fix no-std build in readers.rs

Signed-off-by: Joel Dice <[email protected]>

rename `error` to `error-context` per latest spec

Signed-off-by: Joel Dice <[email protected]>

rename `error` to `error-context` per latest spec (part 2)

Also, parse string encoding and realloc from encoded `error-context.new` and
`error-context.debug-string` names.

Signed-off-by: Joel Dice <[email protected]>

add `wast` support for parsing async canon opts

And add tests/local/component-model-async/lift-async.wast for round-trip testing
of async lifts (more to come).

Signed-off-by: Joel Dice <[email protected]>

more wast async support and more tests

This also fixes a bug in `wasmprinter` keeping track of core functions.

Signed-off-by: Joel Dice <[email protected]>

more wast async support; add async tests; fix bugs

Signed-off-by: Joel Dice <[email protected]>

more component-model-async tests and fixes

Signed-off-by: Joel Dice <[email protected]>

add `wit-parser` tests for streams, futures, and error-contexts

Signed-off-by: Joel Dice <[email protected]>

add first `wit-component` async test

This required adding a new `wit_parser::decoding::decode_reader_with_features`
function for passing `WasmFeatures` to
`wasmparser::Validator::new_with_features`.

Signed-off-by: Joel Dice <[email protected]>

add more async tests

Signed-off-by: Joel Dice <[email protected]>

add `async-builtins` test for `wit-component`

Signed-off-by: Joel Dice <[email protected]>

add `async-streams-and-futures` test to `wit-component`

Signed-off-by: Joel Dice <[email protected]>

fix stream/future type handling for exported interfaces

Signed-off-by: Joel Dice <[email protected]>

support callback-less (AKA stackful) async lifts

Signed-off-by: Joel Dice <[email protected]>

address review feedback

Signed-off-by: Joel Dice <[email protected]>

* address more review feedback

Signed-off-by: Joel Dice <[email protected]>

* fix test build regression

Signed-off-by: Joel Dice <[email protected]>

* fix more test regressions

Signed-off-by: Joel Dice <[email protected]>

* remove need for `Default` impls in `wast::component::func`

Signed-off-by: Joel Dice <[email protected]>

* fix another test regression

Signed-off-by: Joel Dice <[email protected]>

* minor code simplification

Signed-off-by: Joel Dice <[email protected]>

* require wasmparser `features` feature in wasm-compose

This fixes `cargo check --no-default-features --feature compose`.

Signed-off-by: Joel Dice <[email protected]>

* require wasmparser `features` feature in wit-parser

Signed-off-by: Joel Dice <[email protected]>

* simplify `r#async` parsing code

Signed-off-by: Joel Dice <[email protected]>

* add a bunch of comments to wit-component async code

Signed-off-by: Joel Dice <[email protected]>

* allow importing the same function sync and async

Signed-off-by: Joel Dice <[email protected]>

---------

Signed-off-by: Joel Dice <[email protected]>

…ance#1895)

* Add support for async ABI, futures, streams, and errors

This adds support for encoding and parsing components which use the [Async
ABI](https://github.com/WebAssembly/component-model/blob/main/design/mvp/Async.md)
and associated canonical options and functions, along with the [`stream`,
`future`, and `error`](WebAssembly/component-model#405)
types.

See bytecodealliance/rfcs#38 for more context.

Signed-off-by: Joel Dice <[email protected]>

add wasmparser::WasmFeatures support to wasm-compose

Signed-off-by: Joel Dice <[email protected]>

fix no-std build in readers.rs

Signed-off-by: Joel Dice <[email protected]>

rename `error` to `error-context` per latest spec

Signed-off-by: Joel Dice <[email protected]>

rename `error` to `error-context` per latest spec (part 2)

Also, parse string encoding and realloc from encoded `error-context.new` and
`error-context.debug-string` names.

Signed-off-by: Joel Dice <[email protected]>

add `wast` support for parsing async canon opts

And add tests/local/component-model-async/lift-async.wast for round-trip testing
of async lifts (more to come).

Signed-off-by: Joel Dice <[email protected]>

more wast async support and more tests

This also fixes a bug in `wasmprinter` keeping track of core functions.

Signed-off-by: Joel Dice <[email protected]>

more wast async support; add async tests; fix bugs

Signed-off-by: Joel Dice <[email protected]>

more component-model-async tests and fixes

Signed-off-by: Joel Dice <[email protected]>

add `wit-parser` tests for streams, futures, and error-contexts

Signed-off-by: Joel Dice <[email protected]>

add first `wit-component` async test

This required adding a new `wit_parser::decoding::decode_reader_with_features`
function for passing `WasmFeatures` to
`wasmparser::Validator::new_with_features`.

Signed-off-by: Joel Dice <[email protected]>

add more async tests

Signed-off-by: Joel Dice <[email protected]>

add `async-builtins` test for `wit-component`

Signed-off-by: Joel Dice <[email protected]>

add `async-streams-and-futures` test to `wit-component`

Signed-off-by: Joel Dice <[email protected]>

fix stream/future type handling for exported interfaces

Signed-off-by: Joel Dice <[email protected]>

support callback-less (AKA stackful) async lifts

Signed-off-by: Joel Dice <[email protected]>

address review feedback

Signed-off-by: Joel Dice <[email protected]>

* address more review feedback

Signed-off-by: Joel Dice <[email protected]>

* fix test build regression

Signed-off-by: Joel Dice <[email protected]>

* fix more test regressions

Signed-off-by: Joel Dice <[email protected]>

* remove need for `Default` impls in `wast::component::func`

Signed-off-by: Joel Dice <[email protected]>

* fix another test regression

Signed-off-by: Joel Dice <[email protected]>

* minor code simplification

Signed-off-by: Joel Dice <[email protected]>

* require wasmparser `features` feature in wasm-compose

This fixes `cargo check --no-default-features --feature compose`.

Signed-off-by: Joel Dice <[email protected]>

* require wasmparser `features` feature in wit-parser

Signed-off-by: Joel Dice <[email protected]>

* simplify `r#async` parsing code

Signed-off-by: Joel Dice <[email protected]>

* add a bunch of comments to wit-component async code

Signed-off-by: Joel Dice <[email protected]>

* allow importing the same function sync and async

Signed-off-by: Joel Dice <[email protected]>

---------

Signed-off-by: Joel Dice <[email protected]>

This adds support for generating bindings which use the [Async
ABI](https://github.com/WebAssembly/component-model/blob/main/design/mvp/Async.md)
along with the [`stream`, `future`, and
`error-context`](WebAssembly/component-model#405) types.

By default, normal synchronous bindings are generated, but the user may opt-in
to async bindings for all or some of the imported and/or exported functions in
the target world and interfaces -- provided the default-enabled `async` feature
is enabled.

In addition, we generate `StreamPayload` and/or `FuturePayload` trait
implementations for any types appearing as the `T` in `stream<T>` or `future<T>`
in the WIT files, respectively.  That enables user code to call `new_stream` or
`new_future` to create `stream`s or `future`s with those payload types, then
write to them, read from them, and/or pass the readable end as a parameter to a
component import or return value of a component export.

Note that I've added new `core::abi::Instruction` enum variants to handle async
lifting and lowering, but they're currently tailored to the Rust generator and
will probably change somewhat as we add support for other languages.

This does not include any new tests; I'll add those in a follow-up commit.

Signed-off-by: Joel Dice <[email protected]>

add `async: true` case to Rust `codegen_tests`

This ensures that all the codegen test WIT files produce compile-able bindings
with `async: true` (i.e. all imports lowered and all exports lifted using the
async ABI).  That revealed some issues involving resource methods and
constructors, as well as missing stub support, which I've resolved.

Signed-off-by: Joel Dice <[email protected]>

add codegen tests for futures, streams, and error-contexts

Signed-off-by: Joel Dice <[email protected]>

remove async_support::poll_future

It was both unsafe to use and intended only for testing (and not even good for
that, it turns out).

Signed-off-by: Joel Dice <[email protected]>

add stream/future read/write cancellation support

Also, fix some issues with stream/future payload lifting/lowering which I
_thought_ I had already tested but actually hadn't.

Signed-off-by: Joel Dice <[email protected]>

support callback-less (AKA stackful) async lifts

Signed-off-by: Joel Dice <[email protected]>

This adds support for loading, compiling, linking, and running components which
use the [Async
ABI](https://github.com/WebAssembly/component-model/blob/main/design/mvp/Async.md)
along with the [`stream`, `future`, and
`error-context`](WebAssembly/component-model#405) types.
It also adds support for generating host bindings such that multiple host
functions can be run concurrently with guest tasks -- without monopolizing the
`Store`.

See the [implementation RFC](bytecodealliance/rfcs#38)
for details, as well as [this
repo](https://github.com/dicej/component-async-demo) containing end-to-end smoke
tests.

Signed-off-by: Joel Dice <[email protected]>

fix clippy warnings and bench/fuzzing errors

Signed-off-by: Joel Dice <[email protected]>

revert atomic.wit whitespace change

Signed-off-by: Joel Dice <[email protected]>

fix build when component-model disabled

Signed-off-by: Joel Dice <[email protected]>

bless component-macro expected output

Signed-off-by: Joel Dice <[email protected]>

fix no-std build error

Signed-off-by: Joel Dice <[email protected]>

fix build with --no-default-features --features runtime,component-model

Signed-off-by: Joel Dice <[email protected]>

partly fix no-std build

It's still broken due to the use of `std::collections::HashMap` in
crates/wasmtime/src/runtime/vm/component.rs.  I'll address that as part of the
work to avoid exposing global task/future/stream/error-context handles to
guests.

Signed-off-by: Joel Dice <[email protected]>

maintain per-instance tables for futures, streams, and error-contexts

Signed-off-by: Joel Dice <[email protected]>

refactor task/stream/future handle lifting/lowering

This addresses a couple of issues:

- Previously, we were passing task/stream/future/error-context reps directly to
  instances while keeping track of which instance had access to which rep.  That
  worked fine in that there was no way to forge access to inaccessible reps, but
  it leaked information about what other instances were doing.  Now we maintain
  per-instance waitable and error-context tables which map the reps to and from
  the handles which the instance sees.

- The `no_std` build was broken due to use of `HashMap` in
  `runtime::vm::component`, which is now fixed.

Note that we use one single table per instance for all tasks, streams, and
futures.  This is partly necessary because, when async events are delivered to
the guest, it wouldn't have enough context to know which stream or future we're
talking about if each unique stream and future type had its own table.  So at
minimum, we need to use the same table for all streams (regardless of payload
type), and likewise for futures.  Also, per
WebAssembly/component-model#395 (comment),
the plan is to move towards a shared table for all resource types as well, so
this moves us in that direction.

Signed-off-by: Joel Dice <[email protected]>

fix wave breakage due to new stream/future/error-context types

Signed-off-by: Joel Dice <[email protected]>

switch wasm-tools to v1.220.0-based branch

Signed-off-by: Joel Dice <[email protected]>

check `task.return` type at runtime

We can't statically verify a given call to `task.return` corresponds to the
expected core signature appropriate for the currently running task, so we must
do so at runtime.  In order to make that check efficient, we intern the types.

My initial plan was to use `ModuleInternedTypeIndex` and/or `VMSharedTypeIndex`
for interning, but that got hairy with WasmGC considerations, so instead I added
new fields to `ComponentTypes` and `ComponentTypesBuilder`.

Signed-off-by: Joel Dice <[email protected]>

add `TypedFunc::call_concurrent` and refine stream/future APIs

This implements what I proposed in https://github.com/dicej/rfcs/blob/component-async/accepted/component-model-async.md#wasmtime.  Specifically, it adds:

- A new `Promise` type, useful for working with concurrent operations that require access to a `Store` to make progress.
- A new `PromisesUnordered` type for `await`ing multiple promises concurrently
-`TypedFunc::call_concurrent` (which returns a `Promise`), allowing multiple host->guest calls to run concurrently on the same instance.
- Updated `{Stream|Future}{Writer|Reader}` APIs which use `Promise`

The upshot is that the embedder can now ergonomically manage arbitrary numbers
of concurrent operations.  Previously, this was a lot more difficult to do
without accidentally starving some of the operations due to another one
monopolizing the `Store`.

Finally, this includes various refactorings and fixes for bugs exposed by the
newer, more versatile APIs.

Signed-off-by: Joel Dice <[email protected]>

clean up verbosity in component/func.rs

Signed-off-by: Joel Dice <[email protected]>

snapshot

Signed-off-by: Joel Dice <[email protected]>

implement stream/future read/write cancellation

This required a somewhat viral addition of `Send` and `Sync` bounds for async
host function closure types, unfortunately.

Signed-off-by: Joel Dice <[email protected]>

add `Func::call_concurrent` and `LinkerInstance::func_new_concurrent`

Signed-off-by: Joel Dice <[email protected]>

dynamic API support for streams/futures/error-contexts

Signed-off-by: Joel Dice <[email protected]>

support callback-less (AKA stackful) async lifts

Signed-off-by: Joel Dice <[email protected]>

fix `call_host` regression

Signed-off-by: Joel Dice <[email protected]>

add component model async end-to-end tests

I've ported these over from https://github.com/dicej/component-async-demo

Signed-off-by: Joel Dice <[email protected]>

fix test regressions and clippy warnings

Signed-off-by: Joel Dice <[email protected]>

satisfy clippy

Signed-off-by: Joel Dice <[email protected]>